Machine Observation of the Direction of Human Visual Focus of Attention

People often look at objects and people with which they are likely to interact. The first step for computer systems to adapt to the user and to improve interaction and with people is to locate where they are, and especially the location of their faces on the image. The next step is to track their focus of attention. For this reason, we are interested in techniques for estimating and tracking gaze of people, and in particular the head pose. This thesis proposes a fully automatic approach for head pose estimation independant of the person identity using low resolution images acquired in unconstrained imaging conditions. The developed method is demonstrated and evaluated using a densly sampled face image database. We propose a new coarse-to-fine approach that uses both global and local appearance to estimate head orientation. This method is fast, easy to implement, robust to partial occlusion, uses no heuristiques and can be adapted to other deformable objects. Face region images are normalized in size and slant by a robust face tracker. The resulting normalized imagettes are projected onto a linear auto-associative memory learned using the Widrow-Hoff rule. Linear auto-associative memories require very few parameters and offer the advantage that no cells in hidden layers have to be defined and class prototypes can be saved and recovered for all kinds of applications. A coarse estimation of the head orientation on known and unknown subjects is obtained by searching the best prototype which matches the current image. We search for salient facial features relevant for each head pose. Feature points are locally described by Gaussian receptive fields normalized at intrinsic scale. These descriptors have interesting properties and are less expensive than Gabor wavelets. Salient facial regions found by Gaussian receptive fields motivate the construction of a model graph for each pose. Each node of the graph can be displaced localy according to its saliency in the image. Linear autoassociative memories deliver a coarse estimation of the pose. We search among the coarse pose neighbors the model graph which obtains the best match. The pose associated with its salient grid graph is selected as the head pose of the person on the image. This method does not use any heuristics, manual annotation or prior knowledge on the face and can be adapted to estimate the pose of configuration of other deformable objects.